Numerically Estimating Derivatives During Simulations

When a function’s value is known at several distinct points, there exist “numerical differentiation” formulas which provide estimates for the first derivative, and with various error bounds. In fact, many times the numerical differentiation formula chosen must be complex to avoid intolerable error as shown below. However, in computer simulations, as well as in real-time programming, while several past data points might be known, as well as the present data, the future is not yet known or computed! This renders those formulas, at least as classically presented, useless for both real-time computing and for simulations. This research began as an inquiry to discover similar formulas usable when only the past and present, but not the future, are known. The author uncovered an extremely obscure corner of numerical analysis, where solutions to these dilemmas are feasible. An extremely simple algorithm can produce suitable formulas, for any number of data points, not only for the first derivative but also higher-order derivatives. Moreover, the data points can be irregularly spaced, and it is straight-forward to calculate the appropriate error terms, and the impact of noisy data. The standard presentation of these methods would require Multivariate Calculus, one or two semesters of Real Analysis, and one semester of Numerical Analysis. However, many computer science graduate students have only Calculus I and Calculus II. This is a novel presentation of this extremely obscure topic, meant not only to publicize it among simluation-oriented computer scientists and those working in real-time computing, but also to be comprehensible to someone who has 2–3 semesters of calculus and nothing more. A rigorous proof of correctness is provided in the appendix, as well as sample code for the algorithm in SAGE— the open source competitor to Maple, Mathematica, Matlab and Magma.